1. Field of the Invention
This invention relates generally to the processing of light fields (including multi-view images) of three-dimensional scenes, for example the processing of light fields of a three dimensional scene captured by a plenoptic imaging system.
2. Description of the Related Art
Light fields have been first introduced in the computer graphics community for representing three-dimensional scenes via multiple views of that scene taken from different viewpoints. In general, the light field of a scene is a seven-dimensional function that contains two-dimensional images (i.e., light field images) of the scene taken from any viewpoint in three-dimensional space, at any wavelength and any time instant. In computer graphics applications, a computer can render the scene from any viewpoint because it has the explicit three-dimensional scene model, including its three-dimensional shape and texture. That is, the computer can render any of the light field images and therefore can also calculate the entire light field of the scene.
Recently, systems have been developed for capturing a four-dimensional light field of three-dimensional scenes. These systems include camera arrays and plenoptic imaging systems. These systems typically capture a four-dimensional light field: two-dimensional images of a scene taken from various viewpoints on a two-dimensional surface (rather than allowing any viewpoint in three-dimensional space), at a certain wavelength (or wavelength band) and time instant. In these systems, the three-dimensional scene information is not explicitly captured. Rather, it is implicitly contained within the pixels of the captured four-dimensional light field.
Extracting three-dimensional information from the four-dimensional light field is an inverse problem. It is a challenging problem because of the high dimensionality of light fields. Dense depth estimation (e.g., estimating depth of each pixel in a scene) is one of those challenging problems, because obtaining a globally smooth and consistent depth map typically requires global optimization, which is usually of prohibitive complexity for such high-dimensional data processing.
Therefore, there is a need for light field processing approaches that efficiently and robustly extract depth and other information from light fields.